Using artificial intelligence to design a product

ABSTRACT

In an embodiment, a method for optimizing computer machine learning includes receiving an optimization goal. The optimization goal is used to search a database of base option candidates (BOC) to identify matching BOCs that at least in part matches the goal. A selection of a selected base option among the matching BOCs is received. Machine learning prediction model(s) are selected based at least in part on the goal to determine prediction values associated with alternative features for the selected base option, where the model(s) were trained using training data to at least identify weight values associated with the alternative features for models. Based on the prediction values, at least a portion of the alternative features is sorted to generate an ordered list. The ordered list is provided for use in manufacturing an alternative version of the selected base option with the alternative feature(s) in the ordered list.

BACKGROUND OF THE INVENTION

Designing a product to meet one or more goals can be difficult. Forexample, it may be challenging to determine what aspect of a productcontributes to the success of the product. Designing for a particularmetric of success may also vary. That is, success may be defined bydifferent metrics or dimensions. Conventional techniques for productdesign typically rely on the intuition of a human designer. However, ahuman designer is often unable to fully assess all data that might becollected about features that could be added to a product or with whicha product may be modified. Conventional computer-aided product designtools are unable to leverage data to guide the designer to optimaldesign decisions. For example, conventional product design tools aretypically unable to be flexible about design or performance goals.

SUMMARY OF THE INVENTION

The present application discloses a method for optimizing computermachine learning that includes the steps of receiving an optimizationgoal, using the optimization goal to search a database of base optioncandidates to identify one or more matching base option candidates thatat least in part matches the optimization goal, receiving a selection ofa selected base option among the matching base option candidates,utilizing one or more machine learning prediction models selected basedat least in part on the optimization goal to determine prediction valuesassociated with alternative features for the selected base option,wherein the one or more machine learning prediction models were trainedusing training data to at least identify machine learning weight valuesassociated with the alternative features for the one or more machinelearning prediction models, based on the prediction values, sorting atleast a portion of the alternative features to generate an ordered listof at least the portion of the alternative features for the selectedbase option, and providing the ordered list for use in manufacturing analternative version of the selected base option with one or more of thealternative features in the ordered list.

The present application further discloses a system for optimizingcomputer machine learning that includes a communications interfaceconfigured to receive an optimization goal and receiving a selection ofa selected base option among matching base option candidates. The systemfurther includes a processor configured to utilize one or more machinelearning prediction models selected based at least in part on theoptimization goal to determine prediction values associated withalternative features for the selected base option, wherein the one ormore machine learning prediction models were trained using training datato at least identify machine learning weight values associated with thealternative features for the one or more machine learning predictionmodels, based on the prediction values, sort at least a portion of thealternative features to generate an ordered list of at least the portionof the alternative features for the selected base option, and providethe ordered list for use in manufacturing an alternative version of theselected base option with one or more of the alternative features in theordered list.

The present application further discloses a computer program product foroptimizing computer machine learning, the computer program product beingembodied in a non-transitory computer readable storage medium andcomprising computer instructions for receiving an optimization goal,using the optimization goal to search a database of base optioncandidates to identify one or more matching base option candidates thatat least in part matches the optimization goal, receiving a selection ofa selected base option among the matching base option candidates,utilizing one or more machine learning prediction models selected basedat least in part on the optimization goal to determine prediction valuesassociated with alternative features for the selected base option,wherein the one or more machine learning prediction models were trainedusing training data to at least identify machine learning weight valuesassociated with the alternative features for the one or more machinelearning prediction models, based on the prediction values, sorting atleast a portion of the alternative features to generate an ordered listof at least the portion of the alternative features for the selectedbase option, and providing the ordered list for use in manufacturing analternative version of the selected base option with one or more of thealternative features in the ordered list.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the followingdetailed description and the accompanying drawings.

FIG. 1 is a flow chart illustrating an embodiment of a process forselecting and providing products.

FIG. 2 is a block diagram illustrating an embodiment of a system forselecting and providing products.

FIG. 3 is a flow chart illustrating an embodiment of a process forsupervised machine learning to train one or more prediction models.

FIG. 4 is a flow chart illustrating an embodiment of a process ofcomputer generated design of a product including base optiondetermination and feature determination.

FIG. 5 is a flow chart illustrating an embodiment of a process foridentifying base options.

FIG. 6 is a flow chart illustrating an embodiment of a process to renderexample products containing alternative features.

FIG. 7 is an example of a GUI for generating a design of a product.

FIG. 8 shows an example GUI of a design tool for generating a design ofa product.

FIG. 9 shows an example GUI of a design tool for generating a design ofa product.

FIG. 10 is an example of a GUI for receiving an optimization goal forgenerating a product.

FIG. 11 is an example of a GUI for displaying one or more base optionsand receiving a selection of a base option for generating a product.

FIG. 12 is an example of a GUI for providing alternative featureselection options and receiving a selection of one or more alternativefeatures for generating a product.

FIG. 13 is an example of a GUI for providing alternative featureselection options and receiving a selection of one or more alternativefeatures for generating a product.

FIG. 14 is an example of a GUI for providing sub-feature featureselection options and receiving a selection of one or more sub-featuresfor generating a product.

FIG. 15 is an example of a design sheet associated with acomputer-generated product.

FIG. 16 is an example of a GUI for generating a preview of a product.

FIG. 17 is a functional diagram illustrating a programmed computersystem for generating a design of a product in accordance with someembodiments.

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as aprocess; an apparatus; a system; a composition of matter; a computerprogram product embodied on a computer readable storage medium; and/or aprocessor, such as a processor configured to execute instructions storedon and/or provided by a memory coupled to the processor. In thisspecification, these implementations, or any other form that theinvention may take, may be referred to as techniques. In general, theorder of the steps of disclosed processes may be altered within thescope of the invention. Unless stated otherwise, a component such as aprocessor or a memory described as being configured to perform a taskmay be implemented as a general component that is temporarily configuredto perform the task at a given time or a specific component that ismanufactured to perform the task. As used herein, the term ‘processor’refers to one or more devices, circuits, and/or processing coresconfigured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention isprovided below along with accompanying figures that illustrate theprinciples of the invention. The invention is described in connectionwith such embodiments, but the invention is not limited to anyembodiment. The scope of the invention is limited only by the claims andthe invention encompasses numerous alternatives, modifications andequivalents. Numerous specific details are set forth in the followingdescription in order to provide a thorough understanding of theinvention. These details are provided for the purpose of example and theinvention may be practiced according to the claims without some or allof these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

Design of a product using artificial intelligence is disclosed. Newproducts are commonly designed to include product features selectedbased on intuition of a designer. However, given technologicaladvancements in machine learning and artificial intelligence as well asbig data analysis capabilities, new products are able to be designed atleast in part using machine learning and artificial intelligence basedon vast amounts of empirical data that would be impossible to analyzewithout the assistance of computers.

In various embodiments, optimizing computer machine learning to generatea design of a product includes receiving an identifier of anoptimization goal. For example, the goal may include design and/orperformance goals. In some embodiments, an example of a performance goalis “designing a new product that will be commercially successful forcustomers in Client Segment 1 for use in the winter season.” A clientsegment may be a shared characteristic among those clients falling intothe client segment. A client segment may also be defined by sharedobserved preferences. An example of a design goal is one or morefeatures that must be included in the design of a product such as“designing a new product that incorporates polka dots.” A productgenerated by the processes described herein may be based on a baseoption, which may be defined by a set of one or more features. In someembodiments, starting from a blank template, one or more features may beadded to define the base option. Using the example of “designing a newproduct that incorporates polka dots,” the base option specifies “polkadots” as an included feature. In some embodiments, starting from thebase option, one or more features may be swapped out and/or added. Forexample, the base option may be a body and/or defined by fitspecifications. The base option may be a style in inventory (e.g., anexisting style), and features may be modified from the base option.

The optimization goal may be used to search a database of base optioncandidates to identify one or more matching base option candidates. Forexample, an inventory of base options is searched to automaticallyidentify candidates that best match the optimization goal. In responseto providing the candidate base options, a selection of a base optionmay be received. The selection may be made by a user (e.g., a humandesigner) from among the matching base options. For example, thisrepresents human cooperation with machine learning and is also known as“human-in-the-loop machine learning.” The selected base option may beutilized as the basis of a resulting artificial intelligence assistedproduct design. For example, one or more attributes/features of the baseoption may be replaced with corresponding alternative feature(s) and/orone or more new alternative attributes/features may be added to the baseoption with the assistance of machine learning/artificial intelligenceto create a product, which is an even better product design thatoptimizes the optimization goal. Thus, by selecting a base option ratherthan starting from a blank slate of product features, already known tobe successful products may be iteratively improved.

To determine what attributes/features to add or replace in a baseoption, one or more machine learning prediction models selected based atleast in part on the optimization goal is utilized to determineprediction values associated with alternative features for the selectedbase option. The one or more machine learning prediction models weretrained using training data to at least identify machine learning weightvalues associated with the alternative features for the one or moremachine learning prediction models. For example, an alternative featureor a combination of features may be provided to a trained machinelearning model to determine a goal optimization prediction score for thefeature or combination of features with respect to the optimization goaland/or the selected base option. Based on the prediction values, atleast a portion of the alternative features may be sorted to generate anordered list of the at least portion of the alternative features for theselected base option. For example, the goal optimization predictionscores of the features may be used to rank the features with respect topredictions on how their inclusion in the base option would positivelyimpact the goal optimization. The ordered list may be provided for usein manufacturing an alternative version of the selected base option withone or more of the alternative features in the ordered list. Forexample, one of the alternative features is selected for inclusion inthe base option due to its high rank in the order and a designspecification that identifies the selected alternative feature may begenerated to be used to manufacture the resultant product, which is analternative version of the selected base option.

FIG. 1 is a flow chart illustrating an embodiment of a process forselecting and providing products. In some embodiments, at least some ofthe products selected and provided to the customer may be productsgenerated according to the processes described herein. The process ofFIG. 1 may be at least in part implemented on one or more components ofsystem 200 shown in FIG. 2. For example, the process may be performed bydata platform 204 and recommendations engine 202 with respect todatabase 206 and design tool 208. In some embodiments, the process ofFIG. 1 is performed by processor 1702 of FIG. 17.

At 102, a request to enroll is received. The request to enroll may bereceived from a potential customer requesting recommendations and/orproducts. The customer may be enrolled with a product selection anddistribution system such as the system of FIG. 2. Upon enrollment, thepotential customer becomes a customer and information about the customermay be stored. Products may be provided to the customer once or on arecurring/subscription basis. Products may be selected for a customerbased on the customer's preferences which may be learned over time. Insome embodiments, the products selected for the customer may beproducts, which are products designed with the assistance of artificialintelligence/machine learning to modify an existing base option/productto optimize an optimization goal. As part of enrollment, the customermay provide information about his or her preferences. For example, thecustomer may provide information directly or indirectly. The informationmay be provided through a personalized app or third party styling ormessaging platforms. This information may be stored in a database suchas database 206 of FIG. 2.

At 104, customer attributes are determined based on the request toenroll. Customer attributes may include objective attributes such asbiographical information, and measurements, and other client segments.Customer attributes may include subjective attributes such aspreferences for style, fit, colors, designers/brands, and budget. Forexample, a customer may rate specific styles, prints, and/or attributesincluding those products in an inventory and products from otherproviders. The information may be collected through third party apps orplatforms such as apps that allow a user to indicate interests and/orshare interest in products with other users. Customer attributes may becollected when a customer enrolls with the system. For example, thecustomer may complete a survey about his or her measurements (height,weight, etc.), lifestyle, and preferences. This information may bestored to a customer profile. Customer attributes may be determined fromsocial media and content created or curated by the customer on thirdparty platforms such as Pinterest®, Instagram®, Facebook®, LinkedIn®,and the like.

When the customer makes purchases and provides feedback on products,customer attributes may be updated. For example, the customer profilemay be updated. The customer may provide feedback in various formatsincluding completing surveys, writing product reviews, making socialmedia posts, and the like regarding one or more products. Productsrecommended to the customer may be adapted to a customer's changingattributes and taste. In one aspect, the customer's taste may be learnedover time by a computer system and/or stylist.

In various embodiments, customer attributes may be determined based ongeneralizations about other users who share characteristics with aparticular customer. Generalizations about groups of customers may bemade from individual customer attributes. Customers may be grouped byany characteristic, including gender, body type, shared preference(e.g., a measure of similarity between clients such as clients'objective or subjective attributes or learned similarity in productpreferences), and the like.

At 106, product options are determined based on the customer attributes.The product options may be determined by processing the customerattributes to select a subset of products from all products in aninventory. The product options may be provided to stylists. In variousembodiments, instead of directly offering all of the product options tothe customer, a stylist first selects products from among the productoptions to provide to the customer.

At 108, a product selection is received from a stylist based on theproduct options. A stylist (e.g., a human user) selects a productselection from the product options. The product selection may then beoffered to a customer. Suppose a customer is looking for blouses. One ormore blouses may be automatically selected from the inventory based onthe customer's attributes. Instead of providing the blouses directly toa customer, a stylist selects a sub-set of blouses to offer to thecustomer. Statistics about the product selection such as whether an itemwas selected to be part of the product selection, when the item wasselected to be part of the product selection, for who/what type ofcustomer was the item selected, etc. can be stored.

At 110, a product selection is provided to a customer. A shipment ofitems may be provided to a customer. The customer may then decide tokeep or return one or more of the items in the shipment. If the customerdecides to keep an item, then the customer purchases the item.Statistics about the items such as whether they were kept or returned,when they were kept or returned, who/what type of customer kept orreturned the item can be stored.

At 112, feedback about the product selection is received. A customer mayprovide feedback about the product selection such as reasons why thecustomer is keeping or not keeping one or more items in the productselection. The feedback may be provided by the customer in variousformats including completing surveys, writing product reviews, makingsocial media posts, and the like. The feedback may be stored andassociated with the customer and/or the item. In various embodiments,the feedback may be used to design products that might appeal to aparticular customer base or meet optimization goals.

At 114, information about the product selection and/or feedback isstored. For example, a tracked sales metric of the item is updated. Insome embodiments, the sales and feedback information associated with theitem may be stored in a database such as database 206 of FIG. 2.

FIG. 2 is a block diagram illustrating an embodiment of a system forselecting and providing products. In some embodiments, the productsselected and provided to the customer may be products generatedaccording to the processes described herein. The example system 200shown in FIG. 2 includes recommendations engine 202, data platform 204,and database 206, and design tool 208. Each of these components may becommunicatively coupled via network 250.

The recommendations engine 202 may be configured to employ adaptivemachine learning to provide recommendations to stylists who select itemsfor customers from an item inventory. For example, the system may use amachine learning trained model to score products. The top scoringproducts may be provided to the stylist. The stylist (e.g., a human)then selects one or more of the top scoring products to be offered to acustomer. The customer may purchase/keep the product and/or providefeedback about the product. The feedback may be used to improve themachine learning training models.

The data platform 204 may be configured to coordinate operation ofrecommendations engine 202, database 206, and design tool 208. Forexample, when data is generated by interaction of a customer, stylist,designer, and/or supplier with system 200, the data platform 204 maydetermine what information is to be stored. For example, the dataplatform may store the data as part of a training data set for machinelearning as further described herein. The data platform may beconfigured to perform the processes described herein, e.g., the processshown in FIG. 3. In various embodiments, design tool 208 may becommunicatively coupled to data platform 204 and the data platform 204may be configured to perform the processes shown in FIGS. 4-6 based oninput received at the design tool 208.

The database 206 may be configured to store information about customers,products, sales data, performance metrics, and machine learning models.Product information may include data associated with a product or groupof products. Product information may include objective attributes of theproduct such as a stock keeping unit (SKU), item type, item property(e.g., color, pattern, material), etc. Product information may includesubjective attributes of the product such as suitability for body types,season, etc. Product attributes may be identified by a human or by amachine. Product information may include a representation of the productsuch as text, image, video, or other form of data. In some embodiments,information about an item may be stored with associated information suchas customer feedback about the item. In some embodiments, informationabout an item may be stored with statistics such as a sales metric(e.g., statistics related to sales of an item or group of items), aninventory metric (e.g., statistics related to inventory such as numberof units in inventory), variety (e.g., a measure of diversity ofinventory and related information such as addressable market). Invarious embodiments, information about an item may be stored with one ormore associated ratings such as style rating (e.g., a measure ofcustomer satisfaction with a style of an item), size rating (e.g., ameasure of an accuracy of the identified size of an item), fit rating(e.g., a measure of customer satisfaction with how well the item fits),quality rating (e.g., a measure of customer satisfaction with quality ofan item), retention measure (e.g., a measure of a likelihood that aproduct leads to a future purchase by a customer), personalizationmeasure (e.g., a measure of customer satisfaction with how well an itemmatches a customer's personality and uniqueness), style grouping measure(e.g., a likelihood that an item is categorized in a particular group),price value rating (e.g., a rating of a value of the item with respectto its price), and the like. In various embodiments, information aboutan item may be stored that scores the style with an aggregate metricthat represents appropriate weighting/value of any or all of thepreceding metrics together.

In various embodiments, the database 206 stores information about howmany units of each item are in the inventory. Supply chain informationmay be stored such as how many units of an item has been ordered, whenthey are expected to be received to replenish a stock of the item, etc.

As described herein, an alternative feature of a base option may beassessed based on its performance with respect to a metric. Thealternative feature may be ranked highly if it is predicted to performwell for a selected metric of an overall optimization goal. Theevaluation metrics may correspond to one or more machine learning modelsthat quantify an evaluation metric value for different sets of featuresto evaluate whether replacement/addition of the alternative feature inthe set of features better achieves an optimization goal.

In various embodiments, an alternative feature may be selected based atleast in part on collaborative filtering and/or client segmentation. Forexample, an alternative feature may be selected based on a likelihoodthat a style would fall into a cluster (e.g., an addressable market). Todetermine whether a style would fall into a cluster, a set of featuresmaking up the style may be analyzed to determine whether the set wouldcause the style to be categorized in a specific way (e.g., whether itwould be classified in a particular stylistic category. The cluster maybe based on feedback such as feedback from a third party app orplatform.

Machine learning models may include trained models generated from amachine learning process such as the process of FIG. 3. Trained modelsmay be categorized by type such as sales models, inventory models,variety models, etc. For each category of model, a model may begenerated for each of one or more segments such as segments based on oneor more of the following: a target body type, a target seasonality, atarget fiscal quarter, a target customer type or business line (e.g.,women, men, children), a target lifestyle, a target product type (e.g.,blouse, dress, pants), a target style (e.g., edgy, urban, PacificNorthwest), etc. A model may correspond to a particular segment such asa client segment, time period, etc. For example, a first sales model maybe for the sales performance of a product for a group of customers witha first body type such as petite and a second sales model may be for theperformance of a product for a group of customers with a second bodytype such as tall.

In some embodiments, a machine learning trained sales model can beutilized to predict a sales of a product with features indicated to themodel. The sales model may be trained using past sales data.

In some embodiments, a machine learning trained inventory model can beutilized to predict an inventory metric associated with a product withfeatures indicated to the model. The inventory model may be trainedusing sales data, current inventory information, past inventoryinformation, etc.

In some embodiments, a machine learning trained variety model can beutilized to predict a likelihood of whether a product with featuresindicated to the model would be desirable to add to an inventory ofproduct offerings to achieve an ideal distribution of inventory variety.

In one aspect, the variety model may be used to identify the value ofvariety in the inventory even where exactly who the style is for istypically not identifiable. The variety model may measure a product interms of the product globally filling a need. The variety model may helpexpand total addressable market. That is, even if a product does notperform well with a current customer, the product may perform well withpotential/future customers. In another aspect, a customer may preferhaving more color choices even if he or she tends to purchase only onecolor. The customer might like the increased possibility that he or sheis purchasing a unique blouse in his or her color because the blouse isoffered in many color choices. The variety model may be trained usinghigher level indications of desired distribution of inventory variety.For example, machine learning training may have to be utilized todetermine a higher level model for ideal inventory distribution based onhigher level product categories, and the variety model is trained usingthis higher level model to determine a model for ideal inventorydistribution based on product features. For example, a particular blousestyle may be offered in three colors. Although one of the colors may notsell as well as the other two colors, providing the third color as anoption may represent value in the inventory

In various embodiments, other models may be utilized. Example modelsinclude models for style rating, size rating, fit rating, qualityrating, retention, personalization, style grouping, and price valuerating as further described herein with respect to FIG. 2.

The design tool 208 may be configured to employ adaptive machinelearning to help a designer design items for the customers according tothe customers' tastes. The designed items, which may be a hybrid of abase option and one or more features, may be among the items that astylist can choose to offer to a customer. The design tool may beconfigured to execute the processes described herein to design aproduct, where the product incorporates predicted successful featuresselected to meet an optimization goal, as further described herein. Forexample, a designer may use the design tool 208 to create a product. Theselection of one or more features that make up a product may be based onan optimization goal such as increasing a sales rate (e.g., measured byunits sold), selection rate (e.g., measured by frequency of selection tobe offered for sale), appeal to a particular market segment, performancefor a particular season or quarter, etc., as further described herein.Thus, the product may be a result of a combination of machinelearning/artificial intelligence selected alternative features for abase option, where the alternative features are those featuresautomatically determined to be among the best options to meet anoptimization goal. For example, the alternative features may be rankedaccording to how well each alternative feature meets the goal and theten best features (or other threshold number) may be selected andpresented for use as replacement or additional features for a baseoption to create a product. In other words, a product is a base optioncombined with one or more alternative features or combinations ofalternative features.

In various embodiments, the design tool may be configured to perform theprocesses described herein, e.g., the processes shown in FIGS. 4-6, todesign a product. The design tool 208 may receive input, automaticallymake design recommendations using machine learning/artificialintelligence, and generate output design specifications. The output ofthe design tool may be provided to a supplier to manufacture a productaccording to the specifications of the output.

For example, to at least in part automatically design a product, asystem aggregates data collected from a customer, stylist, and/ordesigner. The data platform may build one or more trained models usingmachine learning processes further described herein. The training datato train the models may be based on behavior of the customer, stylist,and/or the designer as stored over time in the customer database,recommendations database, and/or inventory database. When a designerprovides an optimization goal via the design tool, one or more baseoptions are selected by the system and provided to the designer. Thedesigner may select one of the base options. In response, the system mayutilize the models to automatically identify one or more alternativefeatures (or combinations of alternative features) for the base option.The base option may be modified or augmented based on the alternativefeatures. For example, if the base option is a blouse, an alternativefeature may identify a replacement or type of neckline, sleeve length,hem length, or the like. The base option may also be automaticallyselected by the system based on the processes described herein, e.g.,the process of FIG. 5.

FIG. 3 is a flow chart illustrating an embodiment of a process forsupervised machine learning to train one or more prediction models. Theprocess of FIG. 3 may be at least in part implemented on one or morecomponents of system 200 shown in FIG. 2. In some embodiments, theprocess of FIG. 3 is performed by processor 1702 of FIG. 17. In someembodiments, a designer interacts with the process of FIG. 3 using GUI700 of FIG. 7.

At 302, training data is collected and prepared. In supervised machinelearning, training data may be utilized to train a prediction model toperform predictions based on information “learned” from the trainingdata. The collected data may also include validation data to verify theaccuracy of the trained prediction model.

The training data may be derived from data about items stored indatabase 206 of FIG. 2. Different prediction models can be trained fordifferent prediction model categories or segments. To train each of thedifferent models, different sets of training data can be gatheredspecifically for different models to be trained. For example, pastperformance data associated with a metric to be predicted using aparticular type of model is gathered for various different segments, anddifferent models of the particular type may be trained for each of thevarious different segment combinations.

Trained models to be generated may be categorized by type such as salesmodels, inventory models, variety models, rating models, etc. For eachcategory of a model, a model may be generated for each of one or moreclient segments such as segments based on one or more of the following:a target body type, a target seasonality, a target fiscal quarter, atarget customer type or business line (e.g., women, men, children), atarget lifestyle, a target style (e.g., edgy, urban, pacific northwest)a target product type (e.g., blouse, dress, pants), etc.

In some embodiments, a prediction model of a sales model type can beutilized to predict a success or sales of a product with featuresindicated to the model. Training data gathered to train sales typemodels includes data associated with past sales performance of productsand associated information about the particular products (e.g., featuresof the products) and the particular sales.

In some embodiments, a prediction model of an inventory model type canbe utilized to predict a likelihood of whether a product with featuresindicated to the model will be ultimately offered to a customer.Training data gathered to train inventory type models includes dataassociated with past selection performance (e.g., selection rate by astylist) of products and associated information about the particularproducts (e.g., features of the products) and the particular selections.

In some embodiments, a prediction model of a variety model type (can beutilized to predict a likelihood of whether a product with featuresindicated to the model would be desirable to add to an inventory ofproduct offerings to achieve an ideal distribution of inventory variety.Training data gathered to train variety type models includes dataassociated with higher level indications of desired distribution ofinventory variety. For example, machine learning training may beutilized to determine a higher level model for ideal inventorydistribution based on higher level product categories, and the varietymodel is trained using this higher level model to determine a model forideal inventory distribution based on product features.

At 304, supervised machine learning features and parameters areselected. For example, a user may set control parameters for variousmachine algorithms to be used to train a model. The selection of thefeatures refers to the selection of machine learning features orindividual identifiable properties of an item. The features andparameters may be selected based on objectives for the trained model.Examples of features for a garment product include type (e.g., blouse,dress, pants), silhouette (e.g., a shape of the garment), print (e.g., apattern on a fabric), material, hemline, sleeve, etc. Examples offeatures are described with respect to FIG. 8. Identification offeatures may be received. The selection of features to be utilized inprediction models can be defined at least in part by a human user or atleast in part by automatically being determined. For example, a human orartificial intelligence may define features of the prediction models tobe trained.

In various embodiments, the features may be based at least in part onnatural language processing (NLP). For example, a computer system mayextract information from text according to NLP techniques. Textgenerated by and about customers such as in product reviews, commentforms, social media, emails, and the like may be analyzed by an NLPsystem to determine customer preferences. For example, a customer mayprovide feedback (e.g., text) when they receive an item (e.g., 112 ofFIG. 2). The feedback provided by the customer may be processed with NLPtechniques to extract features. NLP techniques include rule-basedengines, clustering, and classification to make determinations aboutcharacteristics of a product that might be considered a feature.Features may be identified by machine learning or computer vision orNLP, and recommended for inclusion in a product design. In variousembodiments, term frequency-inverse document frequency (TFIDF), latentDirichlet allocation (LDA), colocation analyses, and the like can beused to create lower-dimensional representations of styles or togenerate words or phrases representing styles. Various machine learningmethods can then predict metrics/optimization goals using thesefeatures. Features that predict the optimization goal can then berelated back to representative styles to communicate the concept todesigners and/or manufacturers.

In various embodiments, the features may be selected based at least inpart on computer vision. For example, information about items (such asitems stored in database 206) may have image representations. A computersystem may extract information from images or videos of the informationabout items according to computer vision techniques. Computer visionsystems may recognize objects, object properties, commonalities, orgeneralizations about groups of objects to identify patterns andfeatures of objects. Here, computer vision systems may recognize commonattributes between items and identify them as features. Computer visionsystems may identify a feature of a product that is not describable by ahuman. Using the example of a fabric, a computer vision system mayidentify a particular print that a human might not recognize as being incommon between two garments. In some embodiments, a computer visionsystem may allow quantification of the distance between styles ofvarious prints. In various embodiments, features can be discovered bycreating unique cluster spaces using color values. The color values canbe created by edge detection and defining print scale and contrast. Edgedetection may be used to provide distance metrics between styles byquantifying “busyness” of a pattern. In some embodiments, the colorvalues can be defined by using neural networks, convolution, etc. Insome embodiments, the color values can be extracted from data of animage without neural networks. For example, an optimization goal may bepredicted using latent dimensions and/or principal components of thelatent dimensions or clusters (k-means) within those dimensions. Invarious embodiments, features may be based at least in part on neuralnets. An optimization goal may be predicted using numeric colordescriptions of color labels arising from clustering within RGB or LABcolor space.

At 306, one or more models are trained. The trained model maypredict/determine performance of a product having a set of featuresprovided as input to the trained model. Trained models may becategorized by type such as sales models, inventory models, varietymodels, rating models, etc. For each category of models, a model may begenerated for one or more segments. For example, a trained model mayreceive as input a feature or combination of features and predict/scorea performance metric such as sales metric, inventory metric, varietymetric, style rating, size rating, fit rating, quality rating,retention, personalization, style grouping, and price value rating.

In various embodiments, a plurality of models are trained and each modelcorresponds to a respective performance metric. For example, a salesmodel is trained to determine a sales metric, an inventory model istrained to determine an inventory metric, an inventory model is trainedto determine an inventory metric, and rating models may be used todetermine style rating, size rating, fit rating, quality rating,retention, personalization, style grouping, and price value rating. Themodels may be trained with training data sets, where the training datasets correspond to particular categories and segments.

A sales model may score input with respect to the input's sales metric.The sales model may be utilized to predict what features will have ahigh sales metric rate. For example, training sales models can includeinformation about past sales of products in which the features of theproducts are known. An inventory model may score input with respect tothe input's inventory metric. The inventory model may predict thelikelihood of a feature to be offered to a customer. A variety model mayscore input with respect to its value in providing variety or diversityto inventory. A rating model may score input with respect to itspredicted rating. A model may be specialized for a particular segment bytraining with data for only that segment. For example, the model for aparticular segment may predict whether a particular combination offeatures will sell well for customers of a specific client segment.

The model may be trained according to supervised learning or othermachine learning techniques. In supervised learning, the objective is todetermine a weight of a feature in a function that optimizes a desiredresult, where the function is a representation of the relationshipbetween the features. In a training process, weights associated withfeatures of a model are determined via the training. That is, thecontribution of each feature to a predicted outcome of the combinationof features is determined. In various embodiments, the model may betrained using mixed effects models that take into account severalfeatures, some of which may be non-independent. The model may be trainedby ridge regression that attributes credit to a particular feature.

In some embodiments, when training a model, the attribution of eachfeature to the output of the function is determined. In someembodiments, a feature represents a combination of features. Forexample, an individual feature may have a different weighting when thatfeature is combined with another feature. A feature or set of featuresmay define a base option. As more input is provided to a model, theoutput of the function becomes closer to a target or validation result.

In various embodiments, a model may be evaluated after the model hasbeen trained. The error of a model is the difference between actualperformance and modeled performance. In another aspect, in somesituations, a well-trained model may nevertheless diverge from an actualresult. In this situation, a product may have an aspect that makes theproduct perform better than expected. For example, the product mayperform better than predicted by a trained model. The description of thefactor for success is an aspect. This aspect can be leveraged byincorporating the aspect into products as further described herein.

As a result of training, a trained sales/success model may predict theperformance/success of a set of features combined in a product. Forexample, given two sets of features differing only in color, the modelmay predict that the success is attributable to the difference in thecolor feature. Because the trained model is able to attribute success toa particular alternative feature or combination of alternative features,the predictions by the trained model may be used to identify alternativefeatures to be included in a product to best match the optimizationgoal.

At 308, the trained model(s) are applied. The trained model(s) may beapplied to make a recommendation about features to include in a designof a product. Thus, the trained model(s) may recommend one or morealternative features to augment a base option (e.g., existing productutilized as a design starting point). That is, a product includes one ormore attributes of features identified as desirable to be included inthe product using the trained model. As further described herein, thetrained model may be used to make a recommendation about one or morebase options. For example, a search space of base options may beexplored to select base options that perform well (e.g., according to anoptimization goal) using the trained models. A result of a trained modelmay be weighted and combined with a weighted result of the other trainedmodels to rank alternative features.

FIG. 4 is a flow chart illustrating an embodiment of a process ofcomputer generated design of a product including base optiondetermination and feature determination. The process of FIG. 4 may be atleast in part implemented on one or more components of system 200 shownin FIG. 2. In some embodiments, the process of FIG. 4 is performed byprocessor 1702 of FIG. 17. In some embodiments, a designer interactswith the process of FIG. 4 using GUI 700 of FIG. 7.

At 402, an optimization goal is received. The optimization goal maydescribe the design goal and/or performance goal for a product. Theoptimization goal describes a target result (e.g., design element orperformance) desired to be achieved by a product, and may be used as abasis to evaluate base options and/or alternative features. An exampleof an optimization goal is 860 of FIG. 8. In some embodiments, anexample of a performance goal is “designing a new product that will becommercially successful for customers in a segment for use in the winterseason.” An example of a design goal is one or more features that mustbe included in the design of a product such as “designing a new productthat incorporates polka dots.”

An optimization goal may include one or more goal components. In someembodiments, the goal components may identify one or more features suchas color, print, sleeve length, hem length, etc. In some embodiments,the goal components may identify one or more optimization types such assales metric (e.g., goal of designing a product predicted to achieve thehighest sales), inventory metric (e.g., goal of designing a productpredicted to achieve highest selection rate for presentation tocustomers), variety metric (e.g., goal of designing a product predictedto optimize a distribution of products in an inventory), style rating,size rating, fit rating, quality rating, retention, personalization,style grouping, and price value rating, and the like. The goalcomponents may have target segments. In various embodiments, targetsegments refer to current segments and/or future segments or areas inwhich to expand. Example target segments include target business line(e.g., women, men, children), target product type (e.g., blouse, dress,pants), client segment, seasonality (e.g., Spring/Summer, Fall/Winter),etc. The optimization goal may be evaluated for segments of anoptimization type, e.g., optimizes sales for target customers of acertain client segment and in a target product season. The optimizationgoal may be received via a GUI such as GUI 700 of FIG. 7.

In various embodiments, one or more goal components may be provided by auser. For example, the user may select from among several goal componentoptions. In various embodiments, one or more goal components may bedetermined by parsing a string. For example, the optimization goal maybe entered by a user in an input text box.

At 404, the optimization goal is used to identify one or more baseoption candidates from among possible base options. In some embodiments,base option candidates are presented to a user/designer in a rankedorder associated with their desirability with respect to theoptimization goal. The base option candidates may be selected from amongbase options in: a catalog of products (e.g., the products may beavailable from a supplier or may have been previously offered to acustomer), a group of human-curated base options, and/or amachine-selected group of base options. In various embodiments, a baseoption candidate is an item that was available for sale. For example,the base option candidate may be selected based on sales data for theitem that was previously sold. The base option candidate may be selectedbecause the item sold well for a particular segment identified in theoptimization goal. In various embodiments, a base option candidate isdetermined based on past performance data associated with each of thebase option candidates. For example, a ranked list of best sellers for agiven product segment may be used to select one or more of the bestsellers for inclusion in a list of base option candidates (e.g., thethree best-selling blouses for under $25 may be identified as baseoption candidates).

In various embodiments, a base option candidate is determined byevaluating a divergence between actual performance (e.g., as measured bysales data) and predicted performance of the base option as a product.The predicted performance of the base option may be determined from oneor more trained models such as a model trained by the process of FIG. 3.Suppose a product performs better than expected (e.g., as expectedaccording to a trained model trained by machine learning such as theprocess of FIG. 3). This difference between actual and predictedperformance may be scored and scores associated with different baseoptions candidates may be used to rank and sort the base optioncandidates to aid in the selection of the best base option. For example,a product may score higher if the divergence between the actual andpredicted performance is positively larger. For example, the product maybe ranked higher because some aspect of the product caused the productto actually perform better than predicted.

By using the scoring of divergence of a product that performed betterthan predicted by a trained model, aspects of the product that made itsuccessful may be incorporated into a product by using theattribution/feature causing the divergence between the actualperformance and the predicted performance. The divergence between theactual performance and predicted performance accounts for unspecifiedfeatures that cause the product to perform better than predicted, andunspecified features are not accounted for by models. A largerdivergence value indicates that the product has an attribute that theprediction models were unable to effectively model. By selecting thebase option with the largest positive divergence between actual andpredicted performance as the basis for a product design, the productdesign incorporates positive attributes of the base option that wereunable to be effectively modeled by the prediction models.

Additional example details with respect to selection of a base optionare provided along with FIG. 5.

At 406, a selection of a selected base option is received. The baseoption may be defined by a set of one or more features. The selectionmay be made by a designer/user. For example, a user may select the baseoption via a GUI to serve as a basis of a product design. Example GUIsfor providing base option candidates and receiving a selection are shownin FIGS. 7 and 11.

At 408, one or more trained models to evaluate alternative features forthe selected base option are selected. The trained model(s) may beselected based at least in part on the optimization goal and/or theselected base option. That is, one or more machine learning predictionmodels (also referred to as “trained models”) may be selected based atleast in part on the optimization goal to determine prediction valuesassociated with alternative features for the selected base option. Theone or more machine learning prediction models may have been trainedusing training data to at least identify machine learning weight valuesassociated with the alternative features for the one or more machinelearning prediction models, e.g., according to 306 of FIG. 3.

The trained models may be selected to limit the number of trained modelsused because using more models increases processing time. The trainedmodels most relevant to the optimization goal and/or base option may beselected to maximize prediction accuracy while reducing processing time.In various embodiments, a particular category of trained model (e.g.,sales, inventory, or variety) may be selected based at least in part onthe optimization type. In various embodiments, a particular segment maybe selected based at least in part on the optimization goal component.Depending on the optimization goal, a prediction/result of a trainedmodel may be weighted and weights associated with different models aredetermined. For example, the prediction results of different trainedmodels may be weighted and combined to determine an overall predictionvalue that can be used as a basis to rank alternative features.

At 410, eligible alternative features are identified for the selectedbase option. The eligible alternative features may be selected by a userand/or automatically determined. For example, among all possiblefeatures of a prediction model (e.g., features determined in 304 of FIG.3), a subset of features applicable and eligible for inclusion in thebase option as replacements or additional features are identified. Insome embodiments, the features eligible for the selected base option maybe selected according to manually specified rules or machine learnedmodels. The eligible features include features that can be added orreplaced given the already existing features of the selected baseoption. For example, if a base option is a skirt, the eligible featuresmay include length, silhouette, and fabric. Features such as sleeve andneckline would not be selected because they do not apply to skirts.

At 412, the selected trained model(s) are used to score one or more setsof features. For example, a set of features may be made up of thefeatures of a base option and at least one of the alternative features,where the alternative feature replaces one of the features of the baseoption or is added to the set of features of the base option. As anotherexample, a set of features may be made up of the features of a baseoption and a plurality of the alternative features, where the pluralityof alternative features replaces some of the features of the base optionor are added to the set of features of the base option. In variousembodiments, a model trained according to the process of FIG. 3 takesthe set of features and outputs a predicted score of the set offeatures. The model may be used for all possible combinations ofalternative features with the base option to generate predictions foreach of the combinations. Because machine learned weights for featuresare known, a function of the weighted features generates an outputrepresenting a prediction about the combination of features making up aproduct. Using the output about the combination of features, thecombination of features may be ranked. Correspondingly, individualalternative features or combinations of alternative features may beranked. In various embodiments, more than one trained model is utilized.For example, a result of each of the selected trained models may beweighted and a combination of weighted results of several trained modelsis determined as a total score for an alternative feature or combinationof features.

At 414, an ordered list of the alternative features is generated. Invarious embodiments, the ordered list may be generated based on theprediction values by sorting at least a portion of the alternativefeatures to generate an ordered list of at least the portion of thealternative features for the selected base option. Values (e.g., scores)associated with desirability of the different alternative features forthe selected base option with respect to the optimization goaldetermined in 412 are sorted and ranked and the correspondingalternative features are provided in an order list of recommendations(e.g., from best to worst) of alternative features to be utilized tomodify the design of the selected base option. For example, the top tenscoring alternative features or combination of alternative features maybe output in a ranked list. This information may be used to design aproduct, e.g., by incorporating at least some of the features to a baseoption.

At 416, the ordered list of alternative features is provided. Theordered list of alternative features allows a designer or computerprocessor to make modifications to the base option and generate aproduct. The alternative features may be ordered according to theireffects on a prediction result. For example, a highly influentialfeature may be ranked higher than a less influential feature. If anoptimization goal is sales metric, the alternative features may beranked according to how much they would increase sales of a productshould they be incorporated into a product.

In various embodiments, when an alternative feature is selected forinclusion in a product design, examples of already existing productswith that alternative feature that best meet the indicated optimizationgoal may be identified and displayed. For example, because there may besubtle variations in how to implement a particular feature (e.g., typeof stripe pattern), recommendations on exact feature implementations areprovided by locating best goal optimizing examples. An example processis shown in FIG. 6.

In various embodiments, a first alternative feature may be selected forincorporation into a base option. Upon incorporation, the ordered listof alternative features may be re-generated to reflect updates to theranking of alternative features. For example, 406-414 may be repeatedwhere the base option is the base option with the incorporation of theselected alternative feature(s). This may yield a sub-set or differentset of alternative features for incorporation into the base option withthe first incorporated alternative feature. In some embodiments, theranking of alternative features may change in response to a featurebeing incorporated into a base option because features may interact withone another to affect the success of a product. That is, a successfulfeature might not work well with another successful feature such thathaving both features in a product may make the product less successful.For example, long sleeves may not work well with polka dots. As anotherexample, two features (e.g., long-sleeve and short-sleeve) might not becompatible.

In some embodiments, a designer may select a plurality of alternativefeatures from the provided ordered list for incorporation into a baseoption to design a product. The resulting product may be input into atrained model to score the product. For example, a designer may selectseveral combinations of features and base options to generate aplurality of products. Each of the products may be scored using thetrained model(s). The products may then be compared to each other todetermine relative predicted performance.

In various embodiments, a representation of a product may be rendered inresponse to alternative feature selection for incorporation into a baseoption. This may help a designer visualize the product. For example, animage of a base option may be altered to show the incorporatedalternative feature. The image of the base option may be combined withan image of the incorporated alternative feature. For example, if athree-quarter sleeve is replaced with a long sleeve, a picture of a baseoption with a three-quarter sleeve may be updated to a picture of a baseoption with a long sleeve. In various embodiments, the representation ofthe product may be displayed alongside the base option to allow forcomparison. Examples of visual representations of a product are shown inFIGS. 9 and 16.

FIG. 5 is a flow chart illustrating an embodiment of a process foridentifying base options. The process of FIG. 5 may be at least in partimplemented on one or more components of system 200 shown in FIG. 2. Insome embodiments, the process of FIG. 5 is performed by processor 1702of FIG. 17. In some embodiments, at least a portion of the process ofFIG. 5 is included in 404 of FIG. 4.

At 502, one or more components of an optimization goal are determined.The optimization goal may describe the design and/or performance goalsfor a product. The optimization goal describes a target result (e.g.,design element and/or performance) desired to be achieved by a product,and may be used as a basis to evaluate base options and/or alternativefeatures. An example of an optimization goal is 860 of FIG. 8. Anoptimization goal may include one or more goal components. The goalcomponents may identify one or more optimization types such as salesmetric (e.g., goal of designing a product predicted to achieve thehighest sales), inventory metric (e.g., goal of designing a productpredicted to achieve highest selection rate for presentation tocustomers), variety metric (e.g., goal of designing a product predictedto optimize a distribution of products in an inventory), style rating,size rating, fit rating, quality rating, retention, personalization,style grouping, price value rating, and the like. The goal componentsmay have target segments such as target business line (e.g., women, men,children), target product type (e.g., blouse, dress, pants), clientsegment, seasonality (e.g., Spring/Summer, Fall/Winter), etc. Theoptimization goal may be evaluated for segments of an optimization type,e.g., optimizes sales for target customers of a certain client segmentand in a target product season. The optimization goal may be receivedvia a GUI such as GUI 700 of FIG. 7.

The optimization goal allows base options to be compared with eachother. For example, “I want to increase profits” may correspond to agoal to increase the sales rate and increase selection rate. In variousembodiments, one or more goal components may be provided by a user. Forexample, the user may select from among several goal component options.In various embodiments, one or more goal components may be determined byparsing a string. For example, the optimization goal may be entered by auser in an input text box.

At 504, eligible base options are selected. The base option candidatesmay be selected from among base options in: a catalog of products (e.g.,the products may be available from a supplier or may have beenpreviously offered to a customer), a group of human-curated baseoptions, a group of one or more sets of one or more features, and/or amachine-selected group of base options. In various embodiments, a baseoption candidate is an item that was available for sale. For example,the base option candidate may be selected based on sales data for theitem that was previously sold. The base option candidate may be selectedbecause the item sold well for a particular segment identified in theoptimization goal. In various embodiments, a base option candidate isdetermined based on past performance data associated with each of thebase option candidates. For example, a ranked list of best sellers for agiven product segment may be used to select one or more of the bestsellers for inclusion in a list of base option candidates (e.g., thethree best-selling blouses for under $25 may be identified as baseoption candidates). In some embodiments, rather than being a specificitem that was available for sale, a base option is defined by a set ofone or more features. For example, a list of one or more features to beutilized as a starting set of features defines a base option.

Among all possible base options, only base options that meet anidentified segment of the optimization goal may be included in theeligible base options. For example, an indicated segment can narrowpossible choices for eligible base options. A goal may indicate certainsegments and only base options in that segment are selected. Using theexample of a segment of an optimization goal to design a blouse forwomen of a first client segment to be worn in the summer season, onlybase options identified as blouses are included in the eligible baseoptions.

At 506, each of the eligible base options is evaluated with respect toeach of the one or more components of the optimization goal. In variousembodiments, a base option is scored based on performance data. Forexample, the eligible base options may be ranked based on sales data,selection data, and variety data. The performance may be based onhistorical data and/or a measurement of how close the possible baseoption actually performed relative to predicted performance. In someembodiments, a prediction result may be made for a base option based ondata, formulae, models, and/or calculations. For example, a formula maybe selected from a database for a base option to determine an aspect ofthe base option.

In various embodiments, models cannot completely predict a resultbecause it is a prediction and actual performance may be different. Aproduct may include unspecified features that cause the product toperform better than predicted but the unspecified features are notaccounted for by models. If we know a prediction using a model for aparticular base option and the actual true past result, we can comparethem. If the true result is more desirable, then that can be accountedfor by the model not being correct for this product and includes someaspect that cannot be predicted by a model for some reason (the reasoncannot be described). This is factored in to identify a product/baseoption that exhibits positive aspects that cannot be modeled.

In various embodiments, a base option candidate is determined byevaluating a divergence between actual performance (e.g., as measured bysales data) and predicted performance of the base option as a product.The predicted performance of the base option may be determined from oneor more trained models such as a model trained by the process of FIG. 3.Suppose a product performs better than expected (e.g., as expectedaccording to a trained model trained by machine learning such as theprocess of FIG. 3). This difference between actual and predictedperformance may be scored and scores associated with different baseoptions candidates may be used to rank and sort the base optioncandidates to aid in the selection of the best base option. For example,a product may score higher if the divergence between the actual andpredicted performance is positively larger. For example, the product maybe ranked higher because some aspect of the product caused the productto actually perform better than predicted.

By using the scoring of divergence of a base option product thatperformed better than predicted by a trained model, aspects of theproduct that made it successful may be incorporated into a product byusing the attribution/feature causing the divergence between the actualperformance and the predicted performance. The divergence between theactual performance and predicted performance accounts for unspecifiedfeatures that cause the product to perform better than predicted, andunspecified features are not accounted for by models. A largerdivergence value indicates that the product has an attribute that theprediction models were unable to effectively model. By selecting thebase option with the largest positive divergence between actual andpredicted performance as the basis for a product design, the productdesign incorporates positive attributes of the base option that wereunable to be effectively modeled by the prediction models.

One or more of these various scores may be used individually and/orweighted and combined.

At 508, an overall evaluation for each of the possible base options isdetermined. The overall evaluation may be a statically combinedaggregation of the score indicators of each of the goal componentsand/or the score of divergence of a base option that performed betterthan predicted by a trained model

At 510, the eligible base options are ranked based on the respectiveoverall evaluation. The base options may be ranked according to how wellthey performed as measured by historical data and/or modeling of thebase options using one or more trained models. For example, base optioncandidates are presented to a user/designer in a ranked order associatedwith their desirability with respect to the optimization goal.

At 512, one or more candidate base options are identified. The candidatebase options may be those possible base options scoring above athreshold score. The candidate base options may be the top pre-definednumber of possible base options. For example, candidate base options arethe top ten possible base options. The candidate base options may beprovided on a GUI for selection by a user. Example GUIs displayingcandidate base options are the GUIs shown in FIGS. 7 and 11.

FIG. 6 is a flow chart illustrating an embodiment of a process to renderexample products containing alternative features. The process of FIG. 6may be at least in part implemented on one or more components of system200 shown in FIG. 2. In some embodiments, the process of FIG. 6 isperformed by processor 1702 of FIG. 17. In some embodiments, at least aportion of the process of FIG. 6 is performed after 416 of FIG. 4.

At 602, a selection of an alternative feature is received. For example,a designer may select an alternative feature presented in a list ofalternative features. The alternative features may be provided in 416 ofFIG. 4 and provided on a GUI such as GUI 800 of FIG. 8.

At 604, one or more examples of items having the selected alternativefeature are identified. In various embodiments, when an alternativefeature is selected for inclusion in a product design, examples ofalready existing products with that alternative feature that best meetan indicated optimization goal may be identified and displayed. Forexample, because there may be subtle variations in how to implement aparticular feature (e.g., type of stripe pattern), recommendations onexact feature implementations are provided by locating best goaloptimizing examples. In some embodiments, among all possible exampleitems, only items that meet an identified segment of an optimizationgoal and also exhibit the selected alternative feature are identified.In some embodiments, each of the identified item examples is evaluatedwith respect to each of the one or more components of an optimizationgoal. In various embodiments, an item is scored based on known pastperformance data with respect to the optimization goal. For example,example items may be ranked based on sales data, selection data, andvariety data. In various embodiments, example items are at least in partranked based on a divergence between an actual performance (e.g., asmeasured by sales data) associated with an item and a predictedperformance of the item as predicted by one or more prediction modelstrained by the process of FIG. 3.

At 606, examples of items having the alternative feature are provided toa user. For example, the examples are rendered on a GUI such as GUI 900of FIG. 9.

FIG. 7 is an example of a GUI for generating a design of a product. TheGUI 700 may be provided as part of a design tool for designing aproduct. In some embodiments, GUI 700 is used by a designer to design aproduct.

GUI 700 includes input fields accepting user input regarding anoptimization goal (the input fields are collectively referred to asinput section 702), output fields providing one or more responses basedon the received input (the output fields are collectively referred to asoutput section 704), and navigation menu 730. The designer may use thedesign tool to design a product that meets the optimization goal. Forexample, the designer may wish to design a garment that is predicted toperform well (e.g., sell well) among a particular client segment.

The input section 702 in this example receives an optimization goal viadrop down menus 750. Here, the optimization goal includes components:business line (e.g., women, men, children), product type (e.g., blouses,dressed, pants), client segment, fiscal quarter or seasonality (e.g.,Spring/Summer, Fall/Winter), and silhouette (e.g., a shape of thegarment). The goal components may be selected via drop-down menus asshown or by other input methods such as text entry, selection of abutton, and the like. The goal components may be pre-populated with adefault value. Here, default selections are shown: “womens” for businessline, “blouses” for class, “Segment 1” for client segment, “Q4” forfiscal quarter, and “all” for silhouette. The goal components are merelyexamples and other components/options are also possible.

In an alternative embodiment, the optimization goal may be indicated byother selection or input methods. FIG. 10 is an example of a GUI forreceiving an optimization goal for generating a product. In the exampleGUI shown in FIG. 10, a user may select a target client segment byclicking on/touching a target client segment (“Under 30,” “30-50,” or“50 and over”). A user may select a season by clicking on/touching aseason (“Q1,” “Q2,” “Q3,” “Q4”).

Returning to FIG. 7, the input section 702 in this example includesrendering options 740. The rendering options may determine how outputsuch as base options are displayed in output section 704. Here,rendering options include: whether to display only vendors that we arestaying with (e.g., vendors that will be available in the future), brandtype (e.g., in-house brands vs. other brands), a style or variants onthe style, and the number of pages over which to split results. Forexample, results may be displayed on a single page or each page may bedefined to show a pre-determined number of results.

The output section 704 in this example includes base options 710generated from the optimization goal components 750. The base optionsmay be displayed in a ranked order of best matching base optionsselected based on the optimization goal components 750. In this example,eight base options are displayed. For each base option, associatedinformation such as ranking metrics may be displayed. In variousembodiments, a base option may be selected for example when a userclicks on or touches the base option. In response to a user selection ofthe base option, one or more alternative features may be displayed. Anexample process for selecting alternative features for a given baseoption is the process of FIG. 6. In various embodiments, in response toselection of a base option, information corresponding to navigation menu730 item “Style Variant Based Replacement Recs” is automaticallyupdated. An example of output corresponding to the “Style Variant BasedReplacement Recs” is shown in FIG. 8.

In an alternative embodiment, output section 704 includes instructionsto select a recommended base option. FIG. 11 is an example of a GUI fordisplaying one or more base options and receiving a selection of a baseoption for generating a product. In this example, a user selects a baseoption in response to a prompt to select a recommended body (baseoption) for a previously selected client group and time period. Theexample of FIG. 11 corresponds to FIG. 10, in which target clientsegment “Segment 3” and season “Q4” are selected.

In various embodiments, the navigation menu 730 allows a user to viewvarious different output generated from the optimization goal. Here, themenu includes: “Body Recommendations” (e.g., base options), “StyleVariant Based Replacement Recs” (e.g., alternative features), “BestStyle Variants with Feature” (e.g., products), “Feature Success inIsolation,” “Feature Contribution,” “Feature Success By Silhouette,”“Feature Pair Interaction,” and “Inventory Holes.” In variousembodiments, the “Body Recommendations” includes a listing of one ormore base options based on an optimization goal. For example, the BodyRecommendations may be a result of 404 of FIG. 4. In variousembodiments, the “Style Variant Based Replacement Recs” includes alisting of one or more alternative features for a base option selectedfrom the Body Recommendations. For example, the Style Variant BasedReplacement Recs may be a result of 414 of FIG. 4. In variousembodiments, the “Best Style Variants with Feature” includes a previewof a product based on a selected base option from “Base Recommendations”and one or more alternative features selected from “Style Variant BasedReplacement Recs.” For example, Best Style Variants with Feature may berendered after 416 of FIG. 4.

In response to input received via input section 702, output may berendered in output section 704. Different types of output may begenerated based on the input and the information may be grouped anddisplayed. In various embodiments, a user may navigate between differentsections using navigation menu 730. FIG. 7 shows a state of the GUIafter input is received at input section 702 and one or more “BodyRecommendations” are rendered in output section 704. An example of“Style Variant Based Replacement Recs” rendered in output section 704 isshown in FIG. 8. An example of “Best Style Variants with Feature”rendered in output section 704 is shown in FIG. 9.

FIG. 8 shows an example GUI of a design tool for generating a design ofa product. In some embodiments, GUI 800 is used by a designer to designa product. FIG. 8 shows a state of the GUI after input is received at aninput section and one or more “Style Variant Based Replacement Recs” arerendered in an output section.

One or more optimization goal components may be selected via drop downmenus 850. Here, the optimization goal includes components: businessline, class, client segment, and fiscal quarter. An example ofoptimization goal selection/provision is discussed with respect to theoptimization goal components 750 of FIG. 7.

To determine one or more trained models to be used to determinealternative features (e.g., 408 of FIG. 4), input may be received viamodel selection input 854. The model selection input 854 may be providedby a user. In this example, the options for models are: Metric 1, Metric1 and Metric 2, Metric 2, Metric 3, and all three metrics (e.g., Metrics1, 2, and 3). For example, Metric 1 may be a sales model, Metric 2 maybe an inventory model, and Metric 3 may be a variety model. Examples ofa sales model, an inventory model, and a variety model are discussedwith respect to FIG. 4. Although not shown, other models or combinationsof models may be used, including but not limited to style rating, sizerating, fit rating, quality rating, retention, personalization, stylegrouping, price value rating, or an aggregate metric. Alternativefeatures may be determined from the selected model(s) by providing oneor more optimization goal components to the selected model(s). Thetrained model(s) may then output one or more alternative features, wherethe output alternative features best meet the optimization goal. Invarious embodiments, the number of alternative features may bepre-defined (e.g., the top threshold number of features are output). Anexample process of selecting alternative features is shown in FIGS. 4and 6. In other embodiments, other options/combinations of models arepossible.

In some embodiments, output is selected for rendering according toranking options 840. The ranking options 840 may allow users to indicatehow results are displayed in a user interface. For example, results ofprocessing by the trained models (selected via model selection input854) may be processed according to ranking selection via ranking options840. In this example, a user may select whether to display features with“color & print” and “vendor & brand,” and whether to remove the effectof price. The “color & print” option allows the color and printrecommendations to be displayed or not displayed. The “vendor & brand”option allows the vendor and brand recommendations to be displayed ornot displayed. The “price” options allows a user to consider models thatpredict what will satisfy the metric overall or to satisfy the metricrelative to a price-point at which the garment will be sold.

In response to input received via optimization goal components 850,model selection input 854, and/or ranking options 840, output may berendered. Different types of output may be generated based on the inputand the information may be grouped and displayed. In variousembodiments, a user may navigate between different sections usingnavigation menu 830. The output displayed in this example corresponds to“Style Variant Based Replacement Recs.”

In this example, the output includes alternative features 820 for baseoption 814. Here, the base option is identified by its name “StyleVariant 12345” at 852. The alternative features selected for the baseoption may be based at least in part on an optimization goal. Here, theoptimization goal 860 (“Designing for Client Segment 1 and Q4, the topfeatures to replace to boost Metric 1 are”) is displayed with the baseoption 814. The optimization goal 860 is a summary of the componentsselected via optimization goal components 850. In this example, thealternative features are selected in part based on the selected trainingmodel (“Metric 1”).

A representation such as an image/photograph, video, or picture of thebase option may be displayed. Here, image 810 of the base option “StyleVariant 12345” is displayed.

In this example, alternative features 820 includes silhouette, print,hemline, sleeve and type. Each feature may have sub-features. Here,sub-features for silhouette are: three quarter sleeve, sleeveless, andlong sleeve. Sub-features for sleeve type are: roll tab sleeve, capsleeve, spaghetti strap, and other. Sub-features for print are: paisley,floral, and abstract. Sub-features for hemline are: standard, hi lo,curved, and side slit. The alternative features 820 may be listed in anordered/ranked list. Here, “silhouette: three quarter sleeve→sleeveless”is listed first because altering the silhouette and, more specifically,replacing the three quarter sleeve of the base option with a sleevelesssilhouette would boost Metric 1 the most.

FIG. 9 shows an example GUI of a design tool for generating a design ofa product. In some embodiments, GUI 900 is used by a designer to designa product. FIG. 9 shows a state of the GUI after input is received at aninput section and one or more “Base Style Variants with Feature” havinga selected alternative feature or combination of features are renderedin an output section. A “base style variant” refers to a product, whichis a product that is a base option with one or more alternativefeatures.

GUI 900 includes alternative features selection section 920 in which auser may indicate the selected alternative feature to add/modify to abase option to produce a product. In this example, the selectedalternative feature is alternative feature 7 (“print name:Graphic→Abstract”) of alternative features 820 of FIG. 8.

In an alternative embodiment, a user may select one or more alternativefeatures by selecting a button inside of or in addition to using thedrop-down menu in features selection section 920. FIG. 12 is an exampleof a GUI for providing alternative feature selection options andreceiving a selection of one or more alternative features for generatinga product. In the example of FIG. 12, a user selects “Neckline” as thealternative feature in response to a prompt to select some of therecommended feature mutations (alternative features) tailored to aselected body (base option) and time. Here, the neckline of the baseoption is a split neck (performing below average for Client Segment 1),the color of the base option is navy (overstocked given performance insummer months), and the hemline is a hi-lo hem (performs well withsleeveless silhouettes). The description for a feature of the baseoption may be automatically generated based on an assessment of theperformance of that feature. Here, the neckline and the color are bothunderperforming and a reason for underperformance may be provided. Forexample, the navy was overstocked.

In some embodiments, each alternative feature may be displayed with acorresponding graph of that alternative feature compared with otherfeatures of that type. For example, for “Neckline” the graph shows animpact of a metric (“metric impact”) of various neckline types (e.g.,Feature 1 to Feature 9). Each bar in the bar graph represents a metricimpact of a neckline type relative to the selected alternative feature.In some embodiments, the graph may be a basis for a description but notdisplayed on the GUI as shown.

In various embodiments, an alternative feature may include one or moresub-features. FIG. 13 is an example of a GUI for providing alternativefeature selection options and receiving a selection of one or morealternative features for generating a product. In the example of FIG.13, in response to selecting neckline, additional sub-features aredisplayed. Here, a ranked list of neckline replacements is displayed(e.g., Neckline A, Neckline B, Neckline C). The user selects “NecklineB” to replace the neckline of the base option with neckline of typeNeckline B.

In various embodiments, an alternative feature may include one or moresub-features. Sub-features are categorizations of features made at afiner granularity. FIG. 14 is an example of a GUI for providingsub-feature feature selection options and receiving a selection of oneor more sub-features for generating a product. In the example of FIG.14, four blouses with V-neck necklines are displayed. The visualrepresentation of the sub-features may help a designer to visualize thesub-features. Here, the user selects option 2, the neckline (e.g.,Neckline B.2) corresponding to Example 2.

Returning to FIG. 9, GUI 900 includes manufacturing options section 940.Here, the manufacturing options include a brand type selection (both“EB,” an example in-house brand, and “market,” which refers to typicalmarket brands) and a minimum number of shipments. The manufacturingoptions may be used to filter out products. For example, suppose aparticular fabric supplier requires a minimum order exceeding what wouldcorrespond to 25 shipments. Abstract fabric from that supplier would befiltered out and any products using that abstract pattern are not to bedisplayed in section 960.

In response to input received via alternative features selection section920 and manufacturing options section 940, output may be rendered.Different types of output may be generated based on the input and theinformation may be grouped and displayed. In various embodiments, a usermay navigate between different sections using navigation menu 930. Theoutput displayed in this example corresponds to “Base Style Variantswith Feature.”

In this example, the base style variants are generated from thecombination of base option 814 (corresponding image 810) of FIG. 8 andalternative feature 2 (“print name: Graphic→Abstract”) of alternativefeatures 820. That is, a product is base option 814 having thealternative feature of abstract print. Here, eight example products 960,each being the base option 814 with abstract print, are displayed.Additional example products may be displayed on other pages or when auser scrolls down. These example products may be selected from adatabase of products. In various embodiments, the example products aresorted by optimization goal and displayed in ranked order.

In various embodiments, a preview of a product may be updated in realtime as alternative features or combinations of features are selected tobe incorporated into a base option or product. FIG. 16 is an example ofa GUI for generating a preview of a product. In this example, theproduct preview 1610 is rendered based on selections of variousfeatures. Here, the alternative features are neckline 1620, sleeve 1630,and fabric 1640. Each of these alternative features also hassub-features as shown. In this example, sub-feature 1622 is selected,sub-feature 1632 is selected, and a combination of sub-features 1642 and1644 are selected.

In various embodiments, a generated product may be described by a designsheet having specifications for manufacturing the product. For example,a design sheet of a product may be generated by the process shown inFIG. 4. The design sheet may be provided directly to the manufacturer toproduce the product. FIG. 15 is an example of a design sheet associatedwith a computer-generated product. In the example of FIG. 15, theproduct is a blouse with the body of SVID 00000, the neckline of SVID00001, and the hemline of SVID 00002. In addition, designer notes may beprovided. In this example, the notes reflect the optimization goals(Client Segment 1 and over for Q4). In various embodiments, the designsheet may include a visual representation of the product (not shown).FIG. 16 is an example of a visual representation of a product.

FIG. 17 is a functional diagram illustrating a programmed computersystem for generating a design of a product in accordance with someembodiments. As will be apparent, other computer system architecturesand configurations can be used to perform the described productgeneration technique. Computer system 1700, which includes varioussubsystems as described below, includes at least one microprocessorsubsystem (also referred to as a processor or a central processing unit(CPU) 1702). For example, processor 1702 can be implemented by asingle-chip processor or by multiple processors. In some embodiments,processor 1702 is a general purpose digital processor that controls theoperation of the computer system 1700. In some embodiments, processor1702 also includes one or more coprocessors or special purposeprocessors (e.g., a graphics processor, a network processor, etc.).Using instructions retrieved from memory 1710, processor 1702 controlsthe reception and manipulation of input data received on an input device(e.g., image processing device 1706, I/O device interface 1704), and theoutput and display of data on output devices (e.g., display 1718).

Processor 1702 is coupled bi-directionally with memory 1710, which caninclude, for example, one or more random access memories (RAM) and/orone or more read-only memories (ROM). As is well known in the art,memory 1710 can be used as a general storage area, a temporary (e.g.,scratch pad) memory, and/or a cache memory. Memory 1710 can also be usedto store input data and processed data, as well as to store programminginstructions and data, in the form of data objects and text objects, inaddition to other data and instructions for processes operating onprocessor 1702. Also as is well known in the art, memory 1710 typicallyincludes basic operating instructions, program code, data, and objectsused by the processor 1702 to perform its functions (e.g., programmedinstructions). For example, memory 1710 can include any suitablecomputer readable storage media described below, depending on whether,for example, data access needs to be bi-directional or uni-directional.For example, processor 1702 can also directly and very rapidly retrieveand store frequently needed data in a cache memory included in memory1710.

A removable mass storage device 1712 provides additional data storagecapacity for the computer system 1700, and is optionally coupled eitherbi-directionally (read/write) or uni-directionally (read only) toprocessor 1702. A fixed mass storage 1720 can also, for example, provideadditional data storage capacity. For example, storage devices 1712and/or 1720 can include computer readable media such as magnetic tape,flash memory, PC-CARDS, portable mass storage devices such as harddrives (e.g., magnetic, optical, or solid state drives), holographicstorage devices, and other storage devices. Mass storages 1712 and/or1720 generally store additional programming instructions, data, and thelike that typically are not in active use by the processor 1702. It willbe appreciated that the information retained within mass storages 1712and 1720 can be incorporated, if needed, in standard fashion as part ofmemory 1710 (e.g., RAM) as virtual memory.

In addition to providing processor 1702 access to storage subsystems,bus 1714 can be used to provide access to other subsystems and devicesas well. As shown, these can include a display 1718, a network interface1716, an input/output (I/O) device interface 1704, an image processingdevice 1706, as well as other subsystems and devices. For example, imageprocessing device 1706 can include a camera, a scanner, etc.; I/O deviceinterface 1704 can include a device interface for interacting with atouchscreen (e.g., a capacitive touch sensitive screen that supportsgesture interpretation), a microphone, a sound card, a speaker, akeyboard, a pointing device (e.g., a mouse, a stylus, a human finger), aGlobal Positioning System (GPS) receiver, an accelerometer, and/or anyother appropriate device interface for interacting with system 1700.Multiple I/O device interfaces can be used in conjunction with computersystem 1700. The I/O device interface can include general and customizedinterfaces that allow the processor 1702 to send and, more typically,receive data from other devices such as keyboards, pointing devices,microphones, touchscreens, transducer card readers, tape readers, voiceor handwriting recognizers, biometrics readers, cameras, portable massstorage devices, and other computers.

The network interface 1716 allows processor 1702 to be coupled toanother computer, computer network, or telecommunications network usinga network connection as shown. For example, through the networkinterface 1716, the processor 1702 can receive information (e.g., dataobjects or program instructions) from another network, or outputinformation to another network in the course of performingmethod/process steps. Information, often represented as a sequence ofinstructions to be executed on a processor, can be received from andoutputted to another network. An interface card or similar device andappropriate software implemented by (e.g., executed/performed on)processor 1702 can be used to connect the computer system 1700 to anexternal network and transfer data according to standard protocols. Forexample, various process embodiments disclosed herein can be executed onprocessor 1702, or can be performed across a network such as theInternet, intranet networks, or local area networks, in conjunction witha remote processor that shares a portion of the processing. Additionalmass storage devices (not shown) can also be connected to processor 1702through network interface 1716.

In addition, various embodiments disclosed herein further relate tocomputer storage products with a computer readable medium that includesprogram code for performing various computer-implemented operations. Thecomputer readable medium includes any data storage device that can storedata which can thereafter be read by a computer system. Examples ofcomputer readable media include, but are not limited to: magnetic mediasuch as disks and magnetic tape; optical media such as CD-ROM disks;magneto-optical media such as optical disks; and specially configuredhardware devices such as application-specific integrated circuits(ASICs), programmable logic devices (PLDs), and ROM and RAM devices.Examples of program code include both machine code as produced, forexample, by a compiler, or files containing higher level code (e.g.,script) that can be executed using an interpreter.

The computer system shown in FIG. 17 is but an example of a computersystem suitable for use with the various embodiments disclosed herein.Other computer systems suitable for such use can include additional orfewer subsystems. In some computer systems, subsystems can sharecomponents (e.g., for touchscreen-based devices such as smart phones,tablets, etc., I/O device interface 1704 and display 1718 share thetouch sensitive screen component, which both detects user inputs anddisplays outputs to the user). In addition, bus 1714 is illustrative ofany interconnection scheme serving to link the subsystems. Othercomputer architectures having different configurations of subsystems canalso be utilized.

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, the invention is not limitedto the details provided. There are many alternative ways of implementingthe invention. The disclosed embodiments are illustrative and notrestrictive.

What is claimed is:
 1. A method for optimizing computer machinelearning, comprising: receiving an optimization goal; using theoptimization goal to search a database of base option candidates toidentify one or more matching base option candidates that at least inpart matches the optimization goal; receiving a selection of a baseoption among the one or more matching base option candidates based inpart on a ranking of the base option among the one or more matching baseoption candidates, wherein the base option is scored based in part on adivergence between actual performance that includes measured sales dataof the base option and performance that includes expected sales data ofthe base option predicted by one or more machine learning predictionmodels, wherein the base option has a score above a threshold score,wherein the base option and the one or more matching base optioncandidates are ranked based on their corresponding scores; utilizing theone or more machine learning prediction models selected based at leastin part on the optimization goal to determine prediction valuesassociated with alternative features for the base option, wherein theone or more machine learning prediction models were trained usingtraining data to at least identify machine learning weight valuesassociated with the alternative features for the one or more machinelearning prediction models; based on the prediction values, sorting atleast a portion of the alternative features to generate an ordered listof at least the portion of the alternative features for the selectedbase option; and providing the ordered list for use in manufacturing analternative version of the selected base option with one or more of thealternative features in the ordered list.
 2. The method of claim 1,further comprising identifying one or more components of optimizationgoal, wherein the one or more components includes at least one of anoptimization type and a target segment.
 3. The method of claim 1,wherein using the optimization goal to identify one or more matchingbase option candidates is based at least in part on past performancedata associated with the one or more matching base option candidates. 4.The method of claim 1, wherein using the optimization goal to identifyone or more matching base option candidates is based at least in part ona variety metric of the one or more matching base option candidates. 5.The method of claim 1, wherein the database of base option candidatesincludes a catalog of products and the selection of a selected baseoption is made by a user.
 6. The method of claim 1, wherein utilizingthe one or more machine learning prediction models includes selectingtraining data based on the optimization goal.
 7. The method of claim 1,wherein sorting at least a portion of the alternative features includesselecting the one or more machine learning prediction models based onthe optimization goal to predict a set of features including the atleast a portion of the alternative features.
 8. The method of claim 1,wherein utilizing the one or more machine learning prediction modelsincludes determining a combination of at least two features andidentifying an associated machine learning weight value for thecombination of the at least two features.
 9. The method of claim 1,wherein utilizing the one or more machine learning prediction modelsincludes supervised learning of the training data.
 10. The method ofclaim 1, wherein utilizing the one or more machine learning predictionmodels includes determining a role of an alternative feature in apredicted performance of a base option.
 11. The method of claim 1,further comprising selecting the at least the portion of the alternativefeatures based on at least one of natural language processing andcomputer vision, wherein the alternative features are filtered based oneligibility for the selected base option.
 12. The method of claim 1,further comprising: receiving a selection of at least one of thealternative features in the ordered list; identifying one or moreexample base options having the selected at least one of the alternativefeatures; and providing the one or more example base options.
 13. Themethod of claim 1, wherein the optimization goal includes predictedperformance with respect to a segment.
 14. The method of claim 1,wherein an alternative feature is selected for inclusion in the orderedlist of alternative features based at least in part on a sales metric ofa set of features including the alternative feature.
 15. The method ofclaim 1, wherein an alternative feature is selected for inclusion in theordered list of alternative features based at least in part on a ratingmetric of a set of features including the alternative feature.
 16. Themethod of claim 1, wherein an alternative feature is selected forinclusion in the ordered list of alternative features based at least inpart on a variety metric of an inventory having a set of featuresincluding the alternative feature.
 17. The method of claim 1, furthercomprising automatically generating a design of a product, wherein theproduct includes the alternative version of the selected base optionwith one or more of the alternative features in the ordered list.
 18. Asystem for optimizing computer machine learning, comprising: acommunications interface configured to: receive an optimization goal;and receive a selection of a base option among one or more matching baseoption candidates based in part on a ranking of the base option amongthe one or more matching base option candidates, wherein the base optionis ranked based in part on a divergence between actual performance thatincludes measured sales data of the base option and performance thatincludes expected sales data of the base option predicted by one or moremachine learning prediction models, wherein the base option has a scoreabove a threshold score, wherein the base option and the one or morematching base option candidates are ranked based on their correspondingscores; a processor configured to: utilize the one or more machinelearning prediction models selected based at least in part on theoptimization goal to determine prediction values associated withalternative features for the selected base option, wherein the one ormore machine learning prediction models were trained using training datato at least identify machine learning weight values associated with thealternative features for the one or more machine learning predictionmodels; based on the prediction values, sort at least a portion of thealternative features to generate an ordered list of at least the portionof the alternative features for the selected base option; and providethe ordered list for use in manufacturing an alternative version of theselected base option with one or more of the alternative features in theordered list.
 19. A computer program product for optimizing computermachine learning, the computer program product being embodied in anon-transitory computer readable storage medium and comprising computerinstructions for: receiving an optimization goal; using the optimizationgoal to search a database of base option candidates to identify one ormore matching base option candidates that at least in part matches theoptimization goal; receiving a selection of a base option among the oneor more matching base option candidates based in part on a ranking ofthe base option among the one or more matching base option candidates,wherein the base option is ranked based in part on a divergence betweenactual performance that includes measured sales data of the selectedbase option and performance that includes expected sales data of thebase option predicted by one or more machine learning prediction models,wherein the base option has a score above a threshold score, wherein thebase option and the one or more matching base option candidates areranked based on their corresponding scores; utilizing the one or moremachine learning prediction models selected based at least in part onthe optimization goal to determine prediction values associated withalternative features for the selected base option, wherein the one ormore machine learning prediction models were trained using training datato at least identify machine learning weight values associated with thealternative features for the one or more machine learning predictionmodels; based on the prediction values, sorting at least a portion ofthe alternative features to generate an ordered list of at least theportion of the alternative features for the selected base option; andproviding the ordered list for use in manufacturing an alternativeversion of the selected base option with one or more of the alternativefeatures in the ordered list.